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Abstract Body 


Background / Context: 

Teachers’ use of effective classroom management skills such as praise, proactive teaching 
strategies, and non-harsh discipline predicts students’ social-emotional competence, fosters the 
development of emotion regulation, and reduces disruptive behaviors (Pianta, LaParo, Payne, 
Cox, & Bradley, 2002; Webster-Stratton, Reid & Stoolmiller, 2007). Although classroom 
management has also been shown to increase student engagement in learning (Creemers, 1994; 
Stringfield, 1994) there has been little controlled research demonstrating the impact of classroom 
management interventions on student achievement (Department of Education, 2008). Teacher 
training interventions therefore warrant more rigorous evaluation with assessment of specific 
academic effects in addition to investigation of social-emotional and behavioral outcomes. 

Professional development for teachers has historically been fairly didactic in nature (Caret et ah, 
2001; Rose & Church, 1998), which is believed to limit effectiveness and translation to teacher 
practice change (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). One well-established 
program with a more active, collaborative training approach is the Incredible Years Teacher 
Classroom Management Program (lYT). Although positive effects on young children’s social- 
emotional competence have been demonstrated for this program when implemented in 
combination with other Incredible Years programs (Webster-Stratton, Reid, & Hammond, 2001; 
2004; Webster-Stratton, Reid, & Stoolmiller, 2008), the program has not been evaluated as an 
independent intervention or with regard to academic outcomes for early elementary students. 

As with many teacher-directed universal interventions for behavior problems or social-emotional 
development, there may be differential effects for students with different risk levels (Bierman et 
al, 2010). In addition, it is possible that social-emotional learning (SEE) interventions may have 
varying impact based upon students’ developmental levels. Typically, it is believed that such 
interventions may be more effective for younger students, although there may be other 
contextual factors that affect outcomes. Thus, moderator analyses can be informative in 
understanding intervention effects. 

Objective / Research Question: 

In this lES-funded Goal 3 study, we examined the efficacy of lYT for improving (a) classroom 
climate and teacher management skills in grades K-2; and (b) students’ attention, social- 
emotional competence, and achievement as moderated by grade level and socio-economic status. 
We hypothesized that students in younger grade levels and those receiving free/reduced lunch 
would experience the greatest benefit. 

Setting: 

Eleven schools across three rural school districts in the southeastern U.S. participated. Rural 
schools were selected due to their lower than average level of school resources and difficulty 
recruiting and retaining highly qualified teachers, which are factors placing students at greater 
risk of educational failure (Bacolod, 2007). However, participating schools varied greatly with 
regard to student race/ethnicity, poverty and achievement levels (ranging from 16-100% 
free/reduced lunch, 19-80% white, and 41-90% of graders at grade level in reading). 
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Participants: 

Participants included 97 K-2 teachers and 1276 students. Teaehers were primarily female and 
Caueasian (94.8% and 81.4%, respectively), with an average of 1 1.25 years of experienee. 

Thirty percent held a master’s degree. Class sizes ranged from 16 to 24 students (M =19, SD = 
1.87). Students were 51% male and 47% reeeived free/redueed luneh. The student sample was 
ethnieally diverse (54% Caueasian, 22% Hispanie, and 17% Afriean-Ameriean). 

Intervention: 

lYT is a teaeher training program that is part of a comprehensive series of interventions 
ineluding parent, ehild, and teaeher training eomponents that were designed to prevent and treat 
aggressive behavior and eonduet problems in young ehildren aged 3-8 years. Its approaeh 
ineludes validated training methods sueh as video-modeling, behavioral rehearsal of key skills 
through numerous role plays, elassroom praetiee assignments, and teaeher goal setting and self- 
monitoring. lYT is provided in 4-6 monthly full-day workshops (5 in the present study) that 
eover building positive relationships with students and parents, proactive elassroom management 
strategies, effeetive use of ineentives, “eoaebing” students soeial and emotional development, 
teaehing ealm-down and problem-solving, and positive diseipline teehniques sueh as redireetion, 
ignoring, and time out. Workshops are led by two trained eo-leaders with approximately 12-15 
teachers in each group. 

In the present study, the average number of workshop training hours aeross the 5 training days 
for intervention teaehers was 34, with an overall attendance rate of 97%. Eaeh teaeher also 
received two brief eonsultation visits in their elassroom (average of 44 minutes total) and regular 
emails to support implementation of strategies taught. Intervention fidelity was strong, as 
indieated by >85% of workshop objeetives being met and teaeher satisfaetion ratings >6 on a 1-7 
seale, and was supported by regular eonsultation ealls with the intervention developer, Carolyn 
Webster-Stratton. In addition, the PI is a nationally certified trainer in the program. 

Research Design: 

This was a randomized, eontrolled effieaey study using a 4-level (students, teaehers, grade 
levels, schools) cluster-randomized bloek design with treatment assignment oeeurring at level 3 
(grade level within sehool). At least one partieipating grade level per sehool was assigned to the 
intervention eondition. Aeross three eohorts, forty-seven teaehers were assigned to the 
intervention condition (of whom 45 participated) with 598 intervention and 560 control students 
with data available for analysis. Due to varying numbers of teaehers within grade levels aeross 
sehools, distribution of teaehers aeross grade levels was unequal (K=36, 1®*=26, 2"‘^=35). 

Data Collection and Analysis: 

Following teacher consent and randomization at the beginning of the school year, trained 
research assistants blind to randomization status observed each teacher’s classroom for 
approximately 2 hours of instructional time and collected teacher ratings on all students for 
whom parental consent was obtained. These assessments were repeated in the spring, following 
completion of approximately five months of intervention. 

Multilevel modeling in SAS (version 9.2, PROC MIXED) was used to account for nesting of 
students within teacher, grade level, and school, and to model change over time. Covariates 
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included free/reduced lunch, sex and race/ethnicity at the student level; teacher years of 
experience, percent of students in the class on free/reduced lunch and percent of students with 
below-average social competence at the classroom level; and percent free/reduced lunch at the 
school level. Hypothesized moderators of the intervention effect included grade level, student 
free/reduced lunch status, and their interaction. 

Measures 

Observational change in teacher practices was assessed with the CLASS (Pianta & Hamre, 2005) 
and the Teacher Coder Impressions Inventory (TCI; Webster-Stratton, Reid, & Hammond, 

2001). The CLASS is a multi-dimensional standardized instrument that has been widely used in 
early education classrooms and has been associated with variation in students’ achievement and 
social adjustment. Subscales examined for this study include Positive Climate, Negative 
Climate, and Behavior Management. The TCI was developed to align more closely with the lYT 
intervention and has been used primarily in research on the efficacy of lYT. Analyses were 
conducted on the Competent subscale of this measure. Reliability of both measures was strong 
(overall kappa for CLASS=.87; TCI=.90). 

Student social-emotional and behavioral outcomes included teacher ratings of emotion 
regulation, pro-social behavior, and inattention on the Revised Teacher Social Competence scale 
(R-TSC; Conduct Problems Prevention Research Group, 1995) and the Conners’ DSM-IV 
Inattention scale (Conners, 2001). T-scores accounting for differences in age and gender were 
utilized in analyses of this latter measure. 

Academic outcomes included the Academic Competence subscale of the R-TSC scale described 
above and the Star Early Literacy/Reading and Math computerized assessment (STAR) 
published by Renaissance Learning. The Academic Competence scale includes teacher-rated 
items related to setting and achieving goals, solving math problems, reading and answering 
questions, and turning in homework. The STAR is a nationally-normed computerized adaptive 
test in which item difficulty is automatically adjusted to reflect students’ skill levels, and was 
designed for repeated administration in order to assess small changes in skill level (i.e. 
curriculum based measurement). Standard scores used in analyses reflect students’ underlying 
ability level on a continuous vertical scale spanning grade levels, and are a useful measure of 
absolute growth. Students in kindergarten complete the Early Literacy assessment, while older 
students are administered Reading, although if they fail screening items they are given the Early 
Literacy test instead. The Math test is only administered to L* and 2"^^ graders. Due to these 
administration limitations, data are not available for students at all grade levels on both 
measures. 

Findings / Results: 

Teacher scores on CLASS indicators of positive behaviors fall in what is considered the 
moderately high range, with limited room for improvement; scores tapping negative behaviors 
were also quite low. Initial examination of baseline data (see Table 1) also indicates that 
students exhibited average or better scores on the R-TSC and Inattention scale, reflecting a 
normative sample. 
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Baseline equivalence. There were no differences between intervention and control teachers on 
any observational scores at pre-test or other classroom-level variables such as teacher experience 
or class size. Nor were any significant baseline group differences found on student 
demographics or social-emotional or behavioral measures (although Emotion Regulation 
approached significance, p = .08, with higher scores for the intervention group). Students in the 
intervention group were rated as having higher baseline Academic Competence and Math scores 
and lower Early Eiteracy scores (p < .05); however, our statistical models, which focus on the 
prediction of change over time, account for such baseline differences (which are more likely to 
be observed in a cluster-randomized, vs. person-randomized, design). 

Teacher Outcomes. Results indicate a significant main effect of the intervention on Positive 
Climate {b = .77, p = .007), reflecting that teachers in the intervention group were rated as having 
significantly more positive change in classroom climate than control teachers over time. 

Contrary to expectation, for the TCI Competent outcome, a significant Experimental Condition x 
Grade Level interaction effect was observed {b = -10.78, p = .005), such that second grade 
teachers in the control group demonstrated significantly greater change in Competence ratings 
over time than second grade teachers in the intervention group. No significant intervention 
effects were observed for the other two observational variables. 

Student Social-Emotional and Attention Outcomes. Student outcome analyses identified no main 
intervention effects; however, several statistically significant moderated effects emerged. There 
was a significant interaction of intervention, grade level, and free/reduced lunch status {b = -0.42, 
p = .036), such that a beneficial effect of intervention for the R-TSC Emotion Regulation 
outcome was observed for kindergarteners receiving free/reduced lunch (p = .036) but not for 
those who do not receive free/reduced lunch (p = .96; see Eigure 1). A similar pattern of findings 
was observed for the R-TSC Prosocial Behavior outcome {b = -0.91, p = .005; Eigure 2), such 
that kindergarteners who received free/reduced lunch significantly benefited from the 
intervention (p = .0498) but those not receiving free/reduced lunch did not significantly benefit 
(p = .56). Eirst graders benefited significantly from the intervention on this outcome regardless 
of free/reduced lunch status (p = .033). No significant intervention effects were observed for the 
teacher-rated Conners’ scale outcome. 

Student Academic Outcomes. Eor the STAR Reading outcome there was a significant 
Experimental Condition x Grade Level x Time interaction (6 = -119. 13, p = . Oil); in first grade, 
the intervention group demonstrated significantly greater improvement over time in reading than 
the control group (p = .004), whereas in second grade, change in reading achievement was not 
significantly different for the intervention and control groups (p = .67; see Eigure 3). Eor the 
STAR Math outcome, when the variance of the random slope at the classroom level was fixed to 
zero (consistent with the final Reading model), the intervention effect failed to reach 
significance. However, a random slope model was associated with a significant interaction 
effect {b = -46.13, p = .049) in a direction contrary to prediction, such that control students on 
free/reduced lunch showed greater academic growth over time than intervention students (p = 
.02). No significant intervention effects were observed for the kindergarten Early Literacy 
outcome or for teacher ratings of Academic Competence. 
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Discussion and Conclusions: 

Overall, this study demonstrates modest effeets of a relatively low-eost universal teaeher training 
intervention on elassroom elimate, student soeial-emotional outeomes, and reading aehievement. 
It eontributes to the effieaey literature for the Ineredible Years teaeher program, whieh is 
supported for use with at-risk ehildren in early ehildhood settings in eombination with other 
interventions, by showing independent intervention effeets in a large, normative sample of early 
elementary students attending rural sehools. It is also the first study of this program to identify 
speeifie effeets on an aeademie test, supporting the link between SEL programs and 
aehievement. 

Given the high baseline ratings of elassroom elimate and teaeher behavior management skills in 
this sample, it is perhaps not surprising that expeeted intervention effeets were seen on only one 
of four indieators of teaeher praetiee ehange; however, the size of the effeet was large and this 
eonstruet has important implieations for student funetioning. It is also possible that the CLASS 
was not sensitive to ehanges in speeifie behavior management praetiees; future analyses will 
examine teaeher self-report data of strategies to explore this possibility. In general, intervention 
effeets for students were moderated by grade level and free/redueed luneh status in the direetion 
predieted. That is, signifieant benefit was obtained for kindergarteners and l'^* graders on soeial- 
emotional outeomes and for 1®' graders in reading. Thus, ehanges in elassroom elimate appeared 
to translate into positive outeomes in the domains expeeted, at least for some students. 

Two eounter-to-hypothesis findings that emerged are more diffieult to interpret. There was 
evidenee that grade eontrol teaehers beeame more eompetent over time in supporting 
elassroom management and soeial-emotional skills than did 2"^^ grade intervention teaehers, and 
that math aehievement of eontrol students on free/redueed may have inereased more than that of 
intervention students on free/redueed luneh, an effeet that may have been influeneed by the 2"^^ 
grade teaehers. If sueh findings are replieated, further researeh is needed to explore any unique 
teaeher faetors and/or developmental differenees in students that may aeeount for them. 
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Appendix B. Tables and Figures 


Table 1. 

Unadjusted Descriptive Statistics for Teacher and Student Outcome Variables (Means and 
Standard Deviations) 


Measure/Subscale 


Group 




Intervention 

Control 



Pre 

Post 

Pre Post 

Teacher Outcomes 

CLASS 

Positive Climate 
Negative Climate 
Behavior Management 

5.46 (.84) 
1.24 (.55) 
5.33 (.89) 

5.82 (.75) 
1.07 (.39) 
5.45 (.87) 

5.63 (.77) 
1.21 (.37) 
5.32 (.92) 

5.54 (.84) 
1.20 (.42) 
5.38 (.87) 

TCI 

Competent 

4.44 (.50) 

4.70 (.50) 

4.40 (.68) 

4.61 (.60) 

Student Outcomes 

R-TSC 

Emotion Regulation 
Prosocial Behavior 
Academic Competence 

3.64 (.99) 
3.42(1.13) 
3.36 (1.20) 

3.79 (.94) 
3.62 (1.09) 
3.72(1.15) 

3.54 (.96) 
3.23 (1.14) 
3.11 (1.17) 

3.64 (.98) 

3.46(1.15) 

3.66(1.15) 

Conners’ Inattention 

53.28 (12.63) 

51.91 (11.39) 

53.57 (12.74) 

52.08 (11.22) 

STAR Early Eiteracy 
STAR Reading 
STAR Math 

552.40(113.61) 
211.47(139.95) 
393.47 (120.74) 

612.20 (136.74) 
278.06 (241.74) 
450.80 (122.13) 

570.16 (108.30) 648.04 (107.54) 
193.84 (142.43) 248.50 (156.22) 
362.00 (1 12.72) 430.92 (122.12) 


Note: CLASS scores rated 1-7 with higher being better. TCI Harsh and Competent item average scores range from 

0- 5, with higher being better for Competent and lower scores being better for TCI Harsh. R-TSC scores range from 

1- 5, with higher being better. Conners ’ Inattention scores are T-scores based on age and sex. STAR standard 
scores reflect ability on a continuous vertical scale scanning grade levels. 
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Figure 1 . Predicted Change in Emotion Regulation by grade and free/reduced lunch status 



Figure 2. Predicted change in Prosocial Behavior by grade and free/reduced lunch status 
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Figure 3. Predicted change in STAR Reading by grade 
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Figure 4. Predicted change in STAR Math by free/reduced lunch 
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